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In this research paper, we introduce a novel gradient-free modified three-term 
conjugate gradient method designed to solve nonlinear equations subject to con- 
vex constraints. Our approach incorporates the projection scheme, which en- 
hances the effectiveness of the proposed method. Building upon the modified 
three-term conjugate gradient method for solving M-tensor systems and ¢,- 
norm-based nonsmooth optimization problems, our method can be regarded as 
an extension of their technique. By making mild assumptions, we establish the 
theoretical convergence properties of our iterative method. Through extensive 


numerical experiments, we demonstrate that our proposed approach is not only 
highly efficient but also outperforms other existing methods in terms of perfor- 
mance and accuracy. 
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1. INTRODUCTION 

Nonlinear systems of equations frequently arise in various real-life scenarios. Solving such problems 
necessitates the utilization of various methods, each with its own set of strengths and weaknesses. This research 
paper focuses on a specific type of problem: convex constraint nonlinear equations. Our objective is to identify 
a vector e € D that satisfies the given constraints satisfying the property. 


Te) =0. (1) 


The mapping J : D — R” is a continuous nonlinear mapping, and D C R” is a closed convex set. The 
problem of solving nonlinear equations subject to constraints is encountered in various practical applications, 
such as heat transfer problems [1], physics phenomena [2], fuzzy problems [3], and economics problems [4], 
among others. To address this problem, researchers have proposed several methods, including the Newtonian, 
Quasi-Newton, and Levenberg methods, for solving the nonlinear (see [5]]-[8]). These methods exhibit 
appealing characteristics, such as fast convergence and straightforward implementation. However, they are not 
well-suited for solving large-scale nonlinear equations with convex constraints, as they require the computation 
and storage of the Jacobian matrix or its approximation at each iteration. 
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The conjugate gradient (CG) method is a well-known iterative technique commonly employed for 
solving large-scale unconstrained optimization problems. Over the years, numerous variations of the conjugate 
gradient method have been developed and extended to address the nonlinear (ip. For instance, 
Ibrahim et al. [9] expanded the hybrid Liu-Storey (LS)-Fetcher—Reeves (FR) conjugate gradient approach 
proposed by Djordjevi’c to tackle using the Solodov and Svaiter projection technique. Notably, 
their method does not involve the storage of matrices at each iteration. Yamashita and Fukushima intro- 
duced a novel three-term conjugate gradient method specifically designed for nonlinear monotone equations 
with convex constraints. They established the global convergence and convergence rate of their method under 
mild assumptions and demonstrated its superior numerical performance compared to other approaches. For 
comprehensive references on methods for solving nonlinear equations with convex constraints, please refer to 
(13)-(16]. Additionally, Liu and Du recently proposed a modified three-term conjugate gradient method 
that proves effective in solving M/-tensor systems and nonsmooth optimization problems incorporating the ¢;- 
norm. They also provided theoretical analysis to support the global convergence of their method. For further 
convergence results concerning the CG method, please consult [18]}-[23}. 

Building upon the work of Liu and Du [17], we present a novel gradient-free method for solving the 
nonlinear (Ip. Our proposed method can be considered as an extension of the approach developed by Liu and 
Du [17]. The global convergence of our method is established, assuming Lipschitz continuity of the underlying 
mapping and a weaker monotonicity condition. 

The remainder of this paper is structured as follows: in section 2, we introduce the gradient-free 
modified three-term conjugate gradient method for solving the constrained nonlinear (ip. Section 3 pro- 
vides the theoretical analysis, establishing the global convergence of the method under mild assumptions. In 
section 4, we present preliminary numerical results to demonstrate the efficiency of our proposed method. 
Finally, we conclude the paper. Throughout this manuscript, the Euclidean norm is denoted by | - |. 


2. METHOD 

Expanding upon the conjugate gradient method developed by Liu and Du for solving m-tensor 
systems and ¢,-norm problems, we present a gradient-free projection method for addressing (ip. Our method 
involves the generation of a trial point k; using the following relation: 


ky = €¢ + O4pe (2) 


and the search direction p; is computed by: 


_ J -JF(es) ift = 0, (3) 
— —J(ez) + per peg —Uiyr-1 ift>0, 
where BT" and VJ; are defined as: 
peers ses = J(e:) — Jeri), Oe : Hienet (4) 


> Yt-1 = J = s 
Ilpe—all? Ilpe—all? 


Lemma 2.1: consider the search direction p,; generated by (Bp. It can be established that p; corresponds to a 
sufficient descent direction. In other words, for all t > 0, the following condition holds: 


I(et)" Pe = —I|J (e+) |. ©) 
Proof: by direct computation, we can see that: 


Tee) "1 (er) Pe- 
(ee) pe = || F(ee) |? + eT coe 2 ae 


T(et) ye-1 
= —||J (ex). 


Definition 2.1: let D C R” be a nonempty closed convex set. Then for any y € R”, its projection onto D, 
denoted by Pp|y], is defined by : 


Pply] := argmin{||y — a|| : 2 € D}. 
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The projection operator Pp has a well-known property, that is, for any y, x € R” the following nonexpansive 
property hold 
I|Poly] — Pola]l| < lly - al. (6) 


In what follows, we state the iterative procedures/steps of our method. 


Algorithm 1 
Input: set an initial point eg € D, the positive constants: Tol > 0, r € (0,1), « € (0,2), a> 0, > 0. Set 
t=0. 
Step 0: compute J(e;). If ||.J(ez)|| < Tol, stop. Otherwise, generate the search direction p; using (3). 
Step 1: determine the step-size a, = max{ar™|m > 0} such that 


T(ex + ape) pe > pas||pe||?. (7) 


Step 2: compute k; = e; + ap, where k; is a trial point. 
Step 3: if k; € D and ||.J(kz)|| = 0, stop. Otherwise, compute the next iterate by 


T(kt)" (ex — ke) 


=P k 
€t+1 D |e — 2x 7 (ke) |)? T(kt)| ; (8) 


Step 4: finally we set ¢ = ¢ + 1 and return to step 1. 


3. CONVERGENCE ANALYSIS 
In this section, we establish the global convergence property of Algorithm|[I] To analyze its conver- 
gence behavior, we impose the following assumptions on the mapping J. Assumption 1: 


i) The solution set of the constrained nonlinear (Ip, denoted by D*, is nonempty. 
ii) The mapping J is Lipschitz continuous on R”. That is, there exists a constant L > O such that 


I| (a) — J(B)I| < Lila — All Va, 8B € R” (9) 

iii) For any 6 € D* and a € R”, it holds that 
J(a)"(a— 6) > 0. (10) 
Lemma 3.1: consider two sequences {p;} and {e,} generated by Algorithm|]] We can guarantee the 


existence of a step size a, that satisfies the line search for all t > 0. Proof: for any m > 0, suppose 
does not hold for the iterate tp —th, then we have, 


=f (ei + ar™ Dig)” Pto < par” ||, \|?. 
Thus, by the continuity of J and with 0 < r < 1, it follows that by letting m — oo, we have, 
—J(€to)" Pty <0; 
which contradicts (5). 


Lemma 3.2: suppose the sequences {e;} and {k;,} are generated by Algorithm[I]under assumption [3.| 
Then, we can observe the following property: 


at => nsf (11) 


rel| J(ex)|I? 
(L+ p)|lpell? J 


Proof: let d, = a,r~'. Assume a, ¥ a, from (7), &, does not satisfy (7). That is, 


—J(e,+ Gept)’ Pt < ud ||pell?- 
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From (9) and (5), it can be obviously seen that, 


ell T(ex)I? < —Je pe 


= (J(et + Arpt) — T(e)) pt — Jd (eg Gupte)’ De 


< LA; ||py\|? + cele |? 


< Ge (L + p)|\pell?: 
This gives the desired inequality (11). 


Lemma 3.3: assuming assumption 3 holds, let {e;} and {k,} be sequences generated by Algorithm]]] 
For any solution e* within the solution set D*, we have the inequality, 


lleev1 — e*||? < ller — e*||? — 7 llee — hell? (12) 
In addition, {e,} is bounded and: 
S- lle — kell* < +00. (13) 
t=0 


Proof: we start by utilizing the weak monotonicity assumption (assumption 3 (iii)) on the mapping J. Conse- 
quently, for any solution e* € D*, we have the following inequality: 


J (ke)? (eg — e*) > T(ki)7 (ez — Fy). 


the above inequality together with (7) gives: 


I(er + apr)” (ee — ke) > wo? ||pel|? = 0. (14) 


from (6) and (14), we have the following: 


2 


12 J (ke) (et — ke) P 
lees — eI? = [Po [er — 22) 04] — 
J (kt)? (et — ke) * : 
< e * TRIE 3) : 
Te, — t 2 i) (Ee — t . 
= lle, — e* ||? an (Oa) 1a ee) +2 (i =) 
— te, — etl]? — ap (ZR — Be) pep yt re, — ey a g? (Leelee = be) 
malls : ( [7c I ) 7) ceeot at ( 17 eI ) 
~ le. etl? — ato 2) ( 2 — DY” 
=e o 1( 17] ) 
< lle, — e*||?. 


thus, the sequence {||e, — e*||} has a nonincreasing and convergent property. Therefore, this makes {e;} to be 
bounded and therefore the following holds. 


co 
uw S- |ler — kil|* < |leo — e*|? < +00. 
t=0 


Remark 3.1: taking into account of the definition of k; and also by (13), it can be deduced that: 


jim a||pz|| = 0. (15) 
— 00 
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Theorem 3.2: suppose assumption 3 holds. Let {e;} and {k,} be sequences generated by Algorithm|l| then: 
lim inf || J(e1)|| = 0. (16) 


Proof: it can be obviously seen from lemma 3 and remark 3 that the sequences {e,} and {k,} are bounded 
by a positive constant say e, and ky respectively. In addition with the continuity of J, it further implies that 
{||7(ez) ||} is bounded by a constant say wu. Also, by Lipschitz continuity, we have that: 


Ilye—all = || 7(ee) — F(ee-a Il S Lllee — eral] < 2Lep (17) 


Now, suppose is not valid, that is, there exist a constant say s > 0 such that s < ||J(ez)||, ¢ > 0. Then this 
along with implies that: 


Ipil| = cs, Vt > 0. (18) 
From (3), it follows that for all t > 1, 
T(er)? yet J (et)? pr-1 
= ||-J(ez) + = = 
\|pe\| (et) pe—1||2 Pt-1 pea? Yt-1 
I Jee) IIllye-all Il 7 (ee) [IIlpe-1 || 
Sole Peale el 
Ilpe—a |? IlPe—all? 
Il J (ee) I Ilye—a | 
= ||J(ee)|| + 2— 7 — 
Ilpe—-al 


<ut4L— 44. 
CS 


From (11). we have: 


(L + p)|lpell? 


res? 
> max ¢ acs, ———— 7 > 0, 
(L+p)y 


which contradicts (15). Hence is valid. 


rel| J(e,)||? 
Paes nsf i ft 


4. NUMERICAL EXPERIMENTS 

In this section, we assess the performance of Algorithm|]](referred to as efficient method of three-term 
(EMTT)) using the Dolan and Moré performance profile [24]. The performance profile takes into account the 
number of iterations, the number of function evaluations, and the central processing unit (CPU) running time. 
We evaluate the efficiency of EMTT by applying it to solve several nonlinear monotone functions with convex 
constraints. To analyze its computational efficiency, we compare EMTT with the following algorithms: 


- Xiao and Zhu conjugate gradient method for convex constrained monotone equations (denoted by con- 
jugate gradient descent (CGD) witha = 1, r=0.1, r=1, p= 1074; 

- Liu and Feng derivative-free iterative method for nonlinear monotone equations with convex constraints 
(denoted by projected Dai- Yuan (PDY)) witha = 1, r=0.5, c=1, w=0.01, x =1; 


We note that all codes were coded and implemented in MATLAB environment using: 


- Control parameters: for Algorithm 1, we select a = 1, r = 0.6, uw = 107*, 2 = 1.8, Tol = 10~°. As for 
Algorithm 1, we select all parameters as in [26]. 


- Dimensions: 1,000, 5,000, 10,000, 50,000, 100, 000. 
- Initial points: e; = (0.1,0.1,--- ,0.1)7, eg = (0.2,0.2,+-- ,0.2)7, e3 = (0.5, 0.5,...,0.5)7, 
eg = (1.2,12,:+ 12) 65 = (1.5, 1.5, << 5 1.5)", eg = (2,2, 25.2)" , er = rand (0,1). 
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The test problems with J = (Ji, Jo,--- , Jn) are given below: 
Problem 1: exponential function [27]. 


+e,—1, for? = 2,3,...,n, 
and D = R". 


Problem 2: modified logarithmic function [27]. 
Ji(e) = In(e; + 1) — , fori = 1,2,3,...,n, 
n 
and D = {e< R” : Se <n,e; > —-1,i=1,2,--- oh 
i=l 
Problem 3: [28]: 


Ji(e) = min (min(|e;|, e7), max(|e;|,e?)) for 7 = 2,3,...,n, 
and D = R’.. 


Problem 4: strictly convex function 1 [27]. 


Ji(e) =e —1, fori = 1,2,...,n, 
and D = R’. 


Problem 5: strictly convex function 2 : 


Ji(e) = + eet 1, for? = 1,2,...,n, 
n 
and D = R‘. 
Problem 6: tridiagonal exponential function ; 
Jy(e) =e- arenes) 


Ji(e) = e; — eoosthler-atertesti)) fori = 2,...,n—1 


? 


In(z) =n e0s(hen—1ten)) 


Problem 7: nonsmooth function : 
Ji(e) = e; — sin |e; — 1|, i = 1, 2,3,...,n, 
and D = {een : 4 <n,e; > —-1,7=1,2,--- | : 
i=l 
Problem 8: the trig exp function [27]: 
Ji (e) = 3e? + 2e2 — 5 + sin(e; — eg) sin(e1 + e2) 
Ji(e) — 3e? + 2€j41 — 54 sin(e; €i41) sin(e; + €i41) + 4e; — e;_1e%- 1% — 3 fori = 2,3,..,.n2—1 


1 
Jn(z) = en—1e°"-! ©" — 4e, — 3, where h = i and D = R’... 


Problem 9 [B31]: 


K= >) 6 a= 


J,(e) = 2d(e; — 1) + 4(t; — 0.25)e;, i= 1,2,3,...,n.and D=R"*. 
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Figure 1 displays the performance profile in terms of the number of iterations. EMTT is depicted 
as the top curve, indicating superior performance. EMTT solves approximately 51% of the test problems 
with fewer iterations, while CGD and PDY solve around 30% and 30% respectively. Similarly, in Figure 2, 
EMTT demonstrates a lower number of function evaluations compared to CGD and PDY. Figure 3 presents 
the performance profile based on CPU time. Once again, EMTT emerges as the top curve, solving the highest 
percentage of problems within a factor 7 of the best time. Specifically, EMTT solves around 61% of the 
test problems with the least CPU time, while CGD and PDY solve approximately 21% and 11% respectively. 
Based on these comparisons, EMTT outperforms CGD and PDY according to the Dolan and Moré metric [24], 
encompassing the number of iterations, the total number of function evaluations, and the CPU time. 


Figure 2. Performance profiles based on number of function evaluations 
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Figure 3. Performance profiles based on CPU time (in seconds) 


5. CONCLUSION 

In this study, we have expanded upon the modified three-term conjugate gradient method originally 
developed for solving M-tensor systems and nonsmooth optimization problems involving the £;-norm. Our 
objective was to apply this method to address nonlinear equations with convex constraints. Our proposed 
method ensures that the search direction satisfies the sufficient descent condition, thereby facilitating efficient 
convergence. Through rigorous analysis, we establish global convergence of the method under the assumption 
that the underlying operator is Lipschitz continuous and satisfies a weaker monotonicity condition. To vali- 
date the effectiveness of our approach, we conducted numerical experiments that demonstrate its efficiency in 
practice. 
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